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Abstract —In this paper, a time-varying distributed convex optimization 
problem is studied for continuous-time multi-agent systems. The objective 
is to minimize the sum of local time-varying cost functions, each of 
which is known to only an individual agent, through local interaction. 
Here the optimal point is time varying and creates an optimal trajectory. 
Control algorithms are designed for the cases of single-integrator and 
double-integrator dynamics. In both cases, a centralized approach is 
first introduced to solve the optimization problem. Then this problem 
is solved in a distributed manner and a discontinuous algorithm based 
on the signum function is proposed in each case. In the case of single¬ 
integrator (respectively, double-integrator) dynamics, each agent relies 
only on its own position and the relative positions (respectively, positions 
and velocities) between itself and its neighbors. A gain adaption scheme 
is introduced in both algorithms to eliminate certain global information 
requirement. To relax the restricted assumption imposed on feasible cost 
functions, an estimator based algorithm using the signum function is 
proposed, where each agent uses dynamic average tracking as a tool 
to estimate the centralized control input. As a trade-off, the estimator 
based algorithm necessitates communication between neighbors. Then 
in the case of double-integrator dynamics, the proposed algorithms are 
further extended. Two continuous algorithms based on, respectively, a 
time-varying and a fixed boundary layer are proposed as continuous 
approximations of the signum function. To account for inter-agent 
collision for physical agents, a distributed convex optimization problem 
with swarm tracking behavior is introduced for both single-integrator 
and double-integrator dynamics. It is shown that the center of the agents 
tracks the optimal trajectory, the connectivity of the agents is maintained 
and inter-agent collision is avoided. Finally, numerical examples are 
included for illustration. 

I. Introduction 

The distributed optimization problem has attracted a significant 
attention recently. It arises in many applications of multi-agent 
systems, where agents cooperate in order to accomplish various tasks 
as a team in a distributed and optimal fashion. We are interested in a 
class of distributed convex optimization problems, where the goal is 
to minimize the sum of local cost functions, each of which is known 
to only an individual agent. 

The incremental subgradient algorithm is introduced as one of 
the earlier approaches addressing this problem 0 In this 
algorithm an estimate of the optimal point is passed through the 
network while each agent makes a small adjustment on it. Recently 
some significant results based on the combination of consensus and 
subgradient algorithms have been published 0-0 For example, 
this combination is used in (4j for solving the coupled optimization 
problems with a fixed undirected graph. A projected subgradient 
algorithm is proposed in j5), where each agent is required to lie in its 
own convex set. It is shown that all agents can reach an optimal point 
in the intersection of all agents’ convex sets even for a time-varying 
communication graph with doubly stochastic edge weight matrices. 

However, all the aforementioned works are based on discrete-time 
algorithms. Recently, some new research is conducted on distributed 
optimization problems for multi-agent systems with continuous-time 
dynamics. Such a scheme has applications in motion coordination 
of multi-agent systems. For example, multiple physical vehicles 
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modelled by continuous-time dynamics might need to rendezvous 
at a team optimal location. In f6j, a generalized class of zero- 
gradient sum controllers is introduced for twice differentiable strongly 
convex functions under an undirected graph. In J7), a continuous¬ 
time version of j5j for directed and undirected graphs is studied, 
where it is assumed that each agent is aware of the convex optimal 
solution set of its own cost function and the intersection of all 
these sets is nonempty. Article 0 derives an explicit expression 
for the convergence rate and ultimate error bounds of a continuous¬ 
time distributed optimization algorithm. In j9j, a general approach 
is given to address the problem of distributed convex optimization 
with equality and inequality constraints. A proportional-integral al¬ 
gorithm is introduced in 1 1~0)-(T2), where GD considers strongly 
connected weight balanced directed graphs and GD extends these 
results using discrete-time communication updates. A distributed 
optimization problem is studied in [13 ] with the adaptivity and finite¬ 
time convergence properties. 

In continuous-time optimization problems, the agents are usually 
assumed to have single-integrator dynamics. However, a broad class 
of vehicles requires double-integrator dynamic models. In addition, 
having time-invariant cost functions is a common assumption in the 
literature. However, in many applications the local cost functions 
are time varying, reflecting the fact that the optimal point could be 
changing over time and creates a trajectory. There are just a few works 
in the literature addressing the distributed optimization problem with 
time-varying cost functions G3-G9- In those works, there exist 
bounded errors converging to the optimal trajectory. For example, 
the economic dispatch problem for a network of power generating 
units is studied in 03- where it is proved that the algorithm is robust 
to slowly time-varying loads. In particular, it is shown that for time- 
varying loads with bounded first and second derivatives the optimiza¬ 
tion error will remain bounded. In GD- a distributed time-varying 
stochastic optimization problem is considered, where it is assumed 
that the cost functions are strongly convex, with Lipschitz continuous 
gradients. It is proved that under the persistent excitation assumption, 
a bounded error in expectation will be achieved asymptotically. In 
|l6| , a distributed discrete-time algorithm based on the alternating 
direction method of multipliers (ADMM) is introduced to optimize a 
time-varying cost function. It is proved that for strongly convex cost 
functions with Lipschitz continuous gradients, if the primal optimal 
solutions drift slowly enough with time, the primal and dual variables 
are close to their optimal values. 

Furthermore, in all articles on distributed optimization mentioned 
above, the agents will eventually approach a common optimal point 
while in some applications it is desirable to achieve swarm behavior. 
The goal of flocking or swarming with a leader is that a group of 
agents tracks a leader with only local interaction while maintaining 
connectivity and avoiding inter-agent collision 03 -PI. Swarm 
tracking algorithms are studied in 118) and (T9| , where it is assumed 
that the leader is a neighbor of all followers and has a constant 
and time-varying velocity, respectively. In |20| , swarm tracking 
algorithms via a variable structure approach are introduced, where the 
leader is a neighbor of only a subset of the followers. In the afore¬ 
mentioned studies, the leader plans the trajectory for the team and the 
agents are not directly assigned to complete a task cooperatively. In 
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ED. the agents are assigned a task to estimate a stationary field while 
exhibiting cohesive motions. Although optimizing a certain team 
criterion while performing the swarm behavior is a highly motivated 
task in many multi-agent applications, it has not been addressed in 
the literature. 

The introduced framework, distributed continuous-time time- 
varying optimization, is of great significance in motion coordination. 
Here, multiple agents cooperatively achieve motion coordination 
while optimizing a time-varying team objective function with only 
local information and interaction. For example, multiple spacecraft 
might need to dock at a moving location distributively with only local 
information and interaction such that the total team performance is 
optimized. Multiple agents moving in a formation or swarm with 
local information and interaction might need to cooperatively figure 
out what optimal trajectory the virtual leader or center of the team 
should follow and that knowledge would help the individual agents 
specify their motions. Furthermore, there is a significant need to use 
distributed optimization in various applications such as economic 
dispatch, internet congestion control, and home automation with 
smart electrical devices. While the studies in the aforementioned 
applications would be simplified by assuming that the changing rate 
of the cost functions or the constraints, is small and hence treated 
as invariant in each time interval, it might be more realistic and 
relevant to explicitly take into account the time-varying nature of 
the cost functions or constraints. As a result, distributed continuous¬ 
time optimization algorithms with time-varying cost functions or 
constraints might serve as continuous-time solvers to figure out the 
optimal trajectory in these applications. 

In this paper, we are faced with several challenges such as: 
1) Having time-varying cost functions, which generally changes 
the problem from finding the fixed optimal point to tracking the 
optimal trajectory. 2) Solving the problem in a distributed manner 
using only local information and local interaction. 3) Solving the 
problem for continuous-time single-integrator and double-integrator 
dynamics, where in the latter case there is only direct control on 
agents’ accelerations. 4) In our algorithms, the signum function is 
employed to compensate for the effect of the inconsistent internal 
time-varying optimization signals among the agents so that the agents 
can reach consensus. As the signum function might cause chattering 
in some applications, it is replaced with continuous approximations 
in some algorithms but additional challenges in analysis would result 
from the replacement. 5) Providing analysis on optimization error 
bounds in scenarios where the agents’ states cannot reach consensus. 
6) The coexistence of the optimization objective and the inherent 
nonlinearity of the swarm tracking behavior. Our preliminary attempts 
for solving the distributed convex optimization problem with time- 
varying cost functions have been presented in ||22j, |23| . 

The remainder of this paper is organized as follows: In Section 
[IT] the notation and preliminaries used throughout this paper are 
introduced. In Section [TIT] the case of single-integrator dynamics is 
studied. In Subsection |III-A[ a centralized approach is introduced. 
Then, in Subsections |III-B| and |III-C[ two discontinuous algorithms 
are proposed to solve the problem in a distributed manner. In Section 

g ie case of double-integrator dynamics is studied. In Subsection 
a centralized algorithm is introduced. Then in Subsections 
and |IV-C| two discontinuous algorithms are defined to solve 
the problem in a distributed manner. Subsections |IV-D| and |IV-E| are 
devoted to extend the proposed discontinuous control algorithms. 
In the discontinuous algorithms, the signum function is used but 
it might cause chattering in some applications. Two continuous 
algorithms are proposed to avoid the chattering effect, where a time- 
varying and a time-invariant approximation of the signum function 
are employed in Subsections |IV-D| and 1 1V- F] respectively. In Section 


[y] the distributed convex optimization problem with swarm tracking 
behavior is studied, where two algorithms for single-integrator and 
double-integrator dynamics are designed in Subsections |V-A| and |V-B[ 
respectively. Finally in Section m numerical examples are given for 
illustration. 


II. NOTATIONS AND PRELIMINARIES 


The following notations are adopted throughout this paper. R + 
denotes the set of positive real numbers. The cardinality of a set S is 
denoted by 151. X denotes the index set {1,..., TV}; The transpose of 
matrix A and vector x are shown as A T and x T , respectively. |[a:|| 
denotes the p-norm of the vector x. We define sig(z)“ = |z| Q sgn(z), 
where z £ R and a > 0. Let l n and 0 n denote the column vectors 
of n ones and zeros, respectively. I n denotes the n x n identity 
matrix. For matrix A and B , the Kronecker product is denoted by 
A<g> B. The gradient and Hessian of function / are denoted by V/ 
and H, respectively. The matrix inequality A > ( >)B , A < ( <)B , 
A > (>)0 and A < (<)0 mean that A — B, B — A, A and —A are 
positive (semijdefinite, respectively. Let A m in[j4] and A m ax[74] denote, 
respectively, the smallest and the largest eigenvalue of the matrix A. 

Let a triplet Q = (V,£,A) be an undirected graph, where V = 
{1,..., N} is the node set and £ C V x V is the edge set, and A = 
[dij] £ R JVxiv is the adjacency matrix. An edge between agents i and 
j, denoted by e = (i,j) £ £, means that they can obtain information 
from each other. In an undirected graph the edges (i.j) and (j. i ) are 
equivalent. We assume ( i , i ) ^ £. The adjacency matrix A is defined 
as aij = dji = 1 if (i,j) £ £ and aij = 0 otherwise. The set of 
neighbors of agent i is denoted by Ni = {j £ V : (j,i) £ £}• A 
sequence of edges of the form (i,j), (j , k), ..., where i,j, k £ V, is a 
path. The graph Q is connected if there is a path from every node to 
every other node. By arbitrarily assigning an orientation for the edges 
in Q, let D = [dj*,] £ R JVx l £ l be the incidence matrix associated with 
Q , where dik = — 1 if the edge leaves node i, dik = 1 if it enters 
node i, and dik = 0 otherwise. Let the Laplacian matrix L = [Z tJ ] £ 
jjJVxat asS ociated with the graph Q be defined as la = j^a a D 

and Uj = —dij for i ^ j. Note that L = DD T . The Laplacian 
matrix L is symmetric positive semidefinite. The undirected graph Q 
is connected if and only if L has a simple zero eigenvalue with the 
corresponding eigenvector ljv and all other eigenvalues are positive 
|24| . When the graph Q is connected, we order the eigenvalues of L as 
Ai [L] = 0 < A 2 [L] < ... < Ajv[L]. Particularly, A 2 [L] is the second 
smallest eigenvalue of the Laplacian matrix L. The above notations 
can also be adopted for time-varying graphs, where Q(t), A(t), D(t) 
and L{t) are, respectively, the undirected graph, the adjacency matrix, 
the incidence matrix and the Laplacian matrix at time t. For the time- 
varying graph Q{t), Xi[L(t)],\/i £ I, is a function of f. As long 
as Q(t) is connected, A 2 [-t/(f)] is uniformly lower bounded above 0 
because there is only a finite number of possible L(t) associated with 

m- 


Lemma 2.1: [251 The second smallest eigenvalue A 2 [L] of the 

Laplacian matrix L associated with the undirected connected graph 
g satisfies A 2 [L\ = min a: T ljv= o !a;# o N ^rf ■ 

Lemma 2.2: Let /(*) : R m — > R be a continuously differentiable 
convex function. The function f(x) is minimized at x* if and only 
if X7 f(x*) = 0 1261. Furthermore, for any strictly convex function 
h(x) : R m —> R, the optimal solution x* , assuming that it exists, is 
unique (271. 

a / Q g 

The symmetric real matrix ( oT 

\ o K 

id only if one of the following conditions hold: 
(i) Q > 0, R - S t Q~ 1 S > 0; or (ii) R>0,Q- SR~ 1 S T > 0. 
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III. Time-Varying Convex Optimization For 
Single-Integrator Dynamics 

Consider a multi-agent system consisting of N physical agents 
with an interaction topology described by the undirected graph Q. It is 
common to adopt single-integrator or double-integrator models. Here, 
suppose that the agents satisfy the continuous-time single-integrator 
dynamics 

±i(t) = Ui(t), iel, (1) 

where Xi(t) £ R m is the position, and u;(f) £ R m is the control 
input of agent i. Note that Xi(t) and Ui{t) are functions of time. 
Later for ease of notation we will write them as Xi and m. A time- 
varying local cost function /; : R m x R + —> R is assigned to agent 
i £ X, which is known to only agent i. The team cost function 
is denoted by Y^,iLi fi{ x >t) and assumed to be convex. Note that 
here only /i(®, i) is required to be convex but not necessarily 

each fi(x,t). Our objective is to design Ui for 0 using only local 
information and local interaction with neighbors such that all agents 
track the optimal state x*(t), where x*(t) is the minimizer of the 
time-varying convex optimization problem 

N 

mjn ( 2 ) 

i=1 


converge to zero when t —> oo. Then by using Lemma |T2| and under 
Assumption 3.1 it is easy to see that x(t) converges to Xo(t), and 
fa will be minimized. ■ 

Remark 3.5: There exist other choices for the control input u 
instead of the one proposed in 0 . For example, u = -rV/o(i, f) — 
Hg 1 (x, £)J^ V/o(a;, f) might be used. In this alternative control 
input, it can be seen that for a time-invariant cost function, 
V fo(x. t) = 0. Hence we will have the well-known gradient 
descent algorithm. For a time-invariant cost function, the proposed 
algorithm 0 will become a Newton algorithm, which is generally 
much faster than the gradient descent algorithm. 

The results from Theorem 13.41 can be extended to minimize the 

and : 


convex function JA =1 /i(®, t). If Assumptions 


3.1 


3.2 


hold, with 


•=C^2 h A x ^)) 1 (' r 5Z v /t( x >*) + X] ^ v /i( x >*)) 


3 = 1 


3 = 1 


3 = 1 


dt 


for 0 , the function fi( x j£) is minimized. Unfortunately, <[ 6 ]» 

is a centralized solution for agents with single-integrator dynamics 
relying on the knowledge of all fi , i £ X. In Subsections |III-B| and 
E 0 will be exploited to propose two algorithms for solving 
the time-varying convex optimization problem for single-integrator 
dynamics in a distributed manner. 


Assumption 3.1: There exists a continuous x*(t) that minimizes 
the team cost function fi( x >t)- 

Because the inverse of the Hessian will be used in our algorithm, 
we need one of the following assumptions to guarantee its existence. 

Assumption 3.2: The function f’X x - t) is twice continu¬ 

ously differentiable with respect to x, with invertible Hessian 

J2f=i H j( x 3), Vx,i. 

Assumption 3.3: Each function fi(x,t) is twice continuously dif¬ 
ferentiable with respect to x, with invertible Hessian Hi(x,t), \/x,t. 

A. Centralized Time-Varying Convex Optimization 

As a first step in this subsection, we focus on the time-varying 
convex optimization problem of 

min fo(x, t), (3) 

X 

where fo : R m x R + — > R is convex in x , for single-integrator 
dynamics 

x = u, (4) 

where x, u £ R m are the system's state and control input, respec¬ 
tively. Next, an algorithm adapted from ]29j will be proposed to solve 
the problem defined by 0 for the system 0- The control input is 
proposed for 0 as 

u = ~Ho 1 (x,t)(TVfo(x,t) + J^V/oO,f)), (5) 

where r > 0 is a positive coefficient; V/o(x,t) and Ho(x,t) are 
respectively, the first and the second derivative of the cost function 
fo(x,t) with respect to x , namely, the gradient and Hessian. 

Theorem 3.4: Suppose that /o satisfies Assumptions ED and |3.2| 
Using 0 for 0 . x(t) converges to the optimal trajectory Xo(t). the 
minimizer of 0 , i.e., lim t _» 00 [a;(f) — Xo (£)] = 0 . 

Proof: Define the positive-definite Lyapunov function candidate 
W = fo(x, t) T V fo(x, t). The derivative of W along the system 
0 with the control input 0 is W = Vf 0 (x,t) T H 0 (x,t)x + 
Vfo(x,t) T -^Vfo(x,t) = -TV/o(i,t) T V/o(i,t). Therefore, 

W < 0 for V/o 7 ^ 0. This guarantees that V/o will asymptotically 


B. Distributed Time-Varying Convex Optimization Using Neighbors’ 
Positions 

In this subsection, we focus on solving the distributed time-varying 
convex optimization problem 0 for agents with single-integrator 
dynamics 0 . Each agent has access to only its own position and 
the relative positions between itself and its neighbors. In some 
applications, the relative positions can be obtained by using only 
agents’ local sensing capabilities, which might in turn eliminate the 
communication necessity between agents. The problem defined in 0 
is equivalent to 

N 

minY^ fi[xi,t) subject to x% = Xj, Vi,j £ 1. (7) 

Xa ‘ J 

i= 1 

Intuitively, the problem is deformed as a consensus problem and a 
minimization problem on the team cost function Here 

the goal is that the states Xi(t),Vi £ X, converge to the optimal 
trajectory x*(t), i.e., 


lim [xi(t) — **(£)] = 0 . 

t —FOO 

The control input is proposed for 0 as 

Ui = - PijSgn(xi - Xj) + 4>i, 

jeNi 


Pij — || x i X j\\l : 3 F ; 

<t>i = - Hr 1 {x i ,t)(yM x i,t) + ^vMxi, t)), 


( 8 ) 


(9) 


where <j>i is an internal signal, /3ij is a varying gain with j3ij( 0 ) = 
{3ji{ 0 ) > 0 , and sgn(-) is the signum function defined componen¬ 
twise. Note that fa depends on only agent i’ s position. Here 0 
is a discontinuous controller. It is worth mentioning that unlike 
continuous or smooth systems, the equilibrium concept of setting 
the right hand equal to zero to find the equilibrium point might not 
be valid for discontinuous systems. Let X = [xf, x%, 
and 4? = [<^f, (j >2 ,..., (Pn] t denote, respectively, the aggregated 
states and the aggregated internal signals of the N agents. We 
also define n = In — Define agent i’s consensus error 

as ex t = Xi — Xe ■ Define the consensus error vector 

ex = (n <g) I m )X. Note that n has one simple zero eigenvalue 
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with ljv as its right eigenvector and has 1 as its other eigenvalue 
with the multiplicity N — 1. Then it is easy to see that ex = 0 if 
and only if Xi = Xj \/i,j £ X. 

Remark 3.6: With the signum function in the proposed algorithms 
in this paper, the right-hand sides of the closed-loop systems are 
discontinuous. Thus, the solution should be investigated in terms of 
differential inclusions by using nonsmooth analysis (30) , (37). How¬ 
ever, since the signum function is measurable and locally essentially 
bounded, the Filippov solutions of the closed-loop dynamics always 
exist. Also the Lyapunov function candidates adopted in the proofs 
hereafter are continuously differentiable. Therefore, the set-valued 
Lie derivative of them is a singleton at the discontinuous points and 
the proofs still hold. To avoid symbol redundancy, we do not use the 
differential inclusions in the proofs. Furthermore, Filippov solutions 
are absolutely continuous curves |30| , which means that the agents’ 
states are continuous functions. 

The remainder of this subsection is devoted to the verification of 
the algorithm 0- In Proposition [I] we will show that the agents reach 
consensus using 0- Then this result will be used in Theorem |3.9| to 
prove that the agents minimize the team cost function as t —► oo. 

Definition 3.7: Defining a[j = atjfiij, a new Laplacian matrix 
L' = [I'ij] £ R JVx/v is introduced, where l' it = j^a a ij and 

l[j = —a'fij for i ^ j. Since a-, = a^, the matrix 7/ is symmetric. 
Similar to the definition of D , D' = [d'y] £ R" xm is the incidence 
matrix associated with L', where d[j = — a- ( if the edge e, leaves 
node i, dij = a[j if it enters node i, and dij = 0 otherwise. Thus, 
7/ can be given by L' = D'D' T . 

Assumption 3.8: With fii defined in 0- there exists a positive 
constant such that \\fii — (f>j\\ 2 < 4>, Vi,j £ X, and Vt. 

Proposition 1: Suppose that the graph Q is connected and As¬ 
sumption |3.8| holds. The system 0 with the algorithm 0 reaches 
consensus, i.e, Xi = Xj,Vi,j £ X, as t —> oo. 

Proof: Using Definition [T7] the closed-loop system 0 with the 
control input 0 can be recast into a compact form as 

X = ~(D' 0 7 m )sgn([L> T 0 I m ]X) + <&, (10) 

where D and D' are defined in Section [jl] and Definition |3.7[ 
respectively. We can rewrite ( | 1 0| > as 

e x = ~ (D 1 0 7 m )sgn([D r 0 Im]e x ) + (n 0 7 m )4>, (11) 


where we have used the fact that UD' = D'. Define the Lyapunov 
function candidate 

W = \e T x ex + ifl E G 8 # - ^) 2 > 

i =1 j€Ni 


where (3 > 0 is to be selected. The time derivative of W along ill: 
can be obtained as 

W = - e x (D' 0 I m )sgn([D T 0 I m \e x ) + ej(n 0 7, n )<f> 

1 N 

+ o E E (fti - 


i= 1 j(E.Ni 

N R 

—EE it 

i=1 j£Ni 
- N N 


\ex, — ex. 


( 12 ) 


2N 


E E( ex * _ e Xj)(tpi - <f>d) 


i= 1 3 = 1 


+ o E E (fc ~ £)(i \ xt - ^ ill) 


=1 ieNi 


= -EE? 


ex, — ex. 


1 


i=i JeNi 
N N 


+ -fx EE( ex - - e *j)(& - fa) - § E E 

i=1 j =1 i=1 j(zNi 


N 


ex, — ex. 


N 


i= 1 j£Ni 

N N 

vEEl 


< 


2 N 


ex,- — ex. 


ex, — ex. 


»=1 3 = 1 


-Iee 

i=1 jENi 


\ex, — ex. 


< 


2 N 


EE I 

»=1 3 = 1 


ex, — ex. 


li-fEE 


ex, — ex,- 


=i ieNi 


where the last inequality holds under Assumption |3.8| Because Q is 
connected, we have 


W < ? max { Y2 j e Xi ~ e Xj 


>-?EE 


< 


2 i 

(N- 1)0 


ex, ~ ex. 


N 

EE 


^ - ex. 


i=ljENi 

Selecting j3 such that ft > ^ N ~ 1 ^ 
,(N- 1)0 P 


2 

N 


i=1 jeNi 
3 N 

111 - 2 E E hxi-ex, 

i=1 jENi 

we have 


W <( 


= ( 


4 2 

(N — l)(j> 


-DEE 

i= 1 jENi 
T 


ex, — ex. 


(13) 


- f3)e x (D 0 7 m )sgn([D T 0 7 m ]ex) 


<( 


(N- 1)0 


- /3) \\{D T 0 7 m )ex 


< ( {N 2 1)(/> - P)\/e T x {DD T 0 7 m )ex 

< ( (JV :^ - P)VW\ l|ex|| 2 < 0, 


whe re in the last inequality the fact that L = DD T and Lemma 
2.1 have been used. Therefore, having W > 0 and W < 0, we can 
conclude that ex £ Coo. By integrating both sides of ( | 13| ), we can 
see that ex £ C- 2 - Now, applying Barbalat’s Lemma (32), we obtain 
that ex will converge to zero asymptomatically and hence the agents’ 
positions reach consensus, i.e, Xi = Xj,\/i,j £ X, as t —> oo. ■ 

Theorem 3.9: Suppose that the graph Q is connected, and Assump¬ 
tions [30 [ 73 ] and [3]8] hold. If Hfixifi) = Hj(xj,t),\/t, Vi,j £ X, 
by employing the algorithm 0 for the system 0, the optimization 
goal 0 is achieved. 

Proof: Define the Lyapunov function candidate 


« iv 

Ww DE v *(*j)) T (E v ^(*i))- 


(14) 


3 = 1 


3 =1 


where W is positive definite with respect to ^_i 
The time derivative of W can be obtained as IT = 
(EE V fi( x d)) T ( EE H3X3+ Ejli Under the as- 

sumption of identical Hessians, we will have 

N N N _ 

w = £ v /i(*j)) r (Hi)£ij + ^r 1 E "S' 

3 = 1 3 =1 3 =1 

On the other hand, by using 0 for the system 0 and summing up 
both sides for j £ X, we know that EyLi x 3 = Ef=i < P:i- Then 
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we can rewrite (B) as W = -(E£=i V/jOj)) T (E^li ^ fi( x i)- 
Therefore, 1C < 0 for EyLi ^fj{ x j) 7 ^ 0- This guarantees that 
EjLi V fj (xj ) will asymptomatically converge to zero. Now, under 
the assumption that Eili fi ( x > () is convex, using Proposition [I] and 
Lemma |T2| it is easy to see that under Assumption ED as t —» 00 
the team cost function E^Li fi{ x ii t) will be minimized, where 07 = 

Xj, Vi,j el. ■ 

Remark 3.10: In 0 each agent i is required to know 
J^Vwhich might be restrictive. However, there are appli¬ 
cations where each agent knows the closed form of its own local cost 
function (e.g., motion control with an optimization objective) or at 
least the agent knows how the cost function is varying with respect to 
time (e.g., home automation). For example, in motion control with an 
optimization objective, it is possible that each agent knows the closed 
form of its local cost function or in home automation smart electrical 
devices need to agree on the total amount of energy consumption that 
maximizes an overall utility function formed by the sum of the utility 
functions of the devices. However, a varying price rate for electricity 
during a day makes the optimization problem time varying. Although 
the price rate of the electricity is varying during the day, it is known 
to the agents beforehand. Hence, calculating^ V t) might not 
be an issue in this application. Furthermore, there are algorithms to 
estimate the derivative of a function by knowing only the value of 
the function at each time t. How to apply the idea to distributed 
continuous-time time-varying optimization is a possible direction for 
our future studies. 

Remark 3.11: Assumption |3.8| intuitively places a bound on the 
Hessians and the changing rates of the gradients of the cost functions 
with respect to t. In Appendix [A] we will show that Assump¬ 
tion |3.8| holds if the cost functions with identical Hessians satisfy 
certain conditions such that the boundedness of I Xi — x 


for 

all t guarantees the boundedness of || S7 fj(xj, t) — Vfi(xi, t)\\ 2 


3 II 2 


and 


Vi,j £ 1, for all t. For 
example, consider the cost functions commonly used for energy 
minimization, e.g., fi(xi,t) = (axt + gi(t)) 2 , where a is a positive 
constant and <?;(() is a time-varying function particularly for agent 
i. For these cost functions, the boundedness of \\xi — Xj \\ 2 for 
all t guarantees the boundedness of || Vfj(xj, t) — V/i(a 7 , f )|| 2 
and - ^V/i(a:i,f )|| 2 , Vi,j € 1, for all t, if 

||< 7 i(t) — gj (t )|| 9 and ||pi(f) — gj{t )|| 2 are bounded. Hence to satisfy 
Assumption 
a bound on 


3.8 


for fi(xi, t) = (axi + gi(t)) , it is sufficient to have 
i T( t) ~ 9j it) || 2 and 11Pi(t) - Pj(f)|| 2 . 

In Subsection |III-C| an estimator-based algorithm is introduced, 
where the assumption on identical Hessians is relaxed. 


C. Estimator-Based Distributed Time-Varying Convex Optimization 

In this subsection, an estimator-based algorithm is designed such 
that each agent calculates § in a distributed manner. To achieve 
this goal, distributed average tracking is used as a tool. Each agent 
generates an estimate of CD- Then a controller is designed such that 
each agent tracks its own generated signal while guaranteeing that 
the agents reach consensus. 

The proposed algorithm for the system 0 has two separate parts, 


the estimator and controller. The estimator part is given by 


£i = a E sgn (wj—Wi) 

, Wi = fi + X7fi(xi,t) 

(16) 




ipi = P E s § n ( 0 f ” di ) 

, 9i = ipi + Hi(xi,t) 

(17) 

3£Ni(t) 

d 

— fi 3“ 77 ^ fi {.Xi , f) 
at 


4>i = v E sgn fe “ ?i )’ 

(18) 


jeJVi(t) 


Si= — 6i 1 (Tw i +s i ), (19) 


where a, j3, 7, and r are positive coefficients to be selected and Ni(t) 
is the set of agent i’s neighbors at time t. The controller part is given 
by 

Ui — E sig (Xi - Xj) v + Si, (20) 

jeJVi(t) 

where sig(-) is defined componentwise and 0 < ij < 1. In 

implementing ( | 1 9| ), 9i can be projected on the space of positive- 
definite matrices, which ensures that 9i remains nonsingular. Also 
, ipi, and <f>i are the internal states of the distributed average tracking 
estimators, where their initial values are such tha(D 

N N N 

E & (0) = E (0) = E (°) = 0- (21) 

3 = 1 j=i 3 =1 

The estimator part (BJ-fB), generates the internal signal for each 
agent and the controller part < |20| ) guarantees consensus. Here the 
separation principle can be applied if the estimator part converges in 
finite time. 

Assumption 3 . 12 : The estimators’ coefficients a,/3, and 7 satisfy 
the following inequalities: a > su Pt || V/i(a 7 , t)|| , [3 > 

Blip* ||£fl'i(*i,i)|| 00 , and 7 > sup t ||^V/ i (x i ,f)|| , Vi el. 

Assumption |3.12| can be satisfied if the partial derivatives of the 
Hessians, the first- and second-order partial derivatives of the gradient 
are bounded. 

Theorem 3 . 13 : Suppose that the graph Q(t) is connected for all t. 
If Assumptions 1 3.1113.2| and |3.12| and the initial condition < |2 1 1 > hold, 
for the system imwith the algorithm {T 6 j-l( 20 ]l, the optimization goal 
0 is achieved. 

Proof: Estimator: It follows from Theorem 2 in | [33) that if As- 
sumption |3.12| holds, then there exists a T > 0 such that for all t >T, 

|| w i - ^EE V/j(a: 3 -,t)|| 2 = 0, ||0i-^Ef=i^.*)|| a = 

0, and \\<li-iJ:f =1 ^fj(xj,t)\\ 2 = 0. Now 

it follows from (19) that for all t > T, Si = 

-(Ef=i Hj(xj,t))-XrY,f = 1 v/j-fo.i) + E?=1 f3 ( x 3,t)), 

Vi e 1. Note that for t > T, 9i is nonsingular without projection 
due to Assumption |3.2| and hence the projection operation simply 
returns 9i itself. Till now we have shown that all agents generate 
the internal signal Si, where Si = Sj,Vi,j e 1, in finite time. 

Controller: Note that V t > T, Si = Sj, Vi,j e 1, denoted as 
S. For t > T, using ( 20 ) for (T). we have 

Xi — ^ . sig(:Ti Xj) 7 ^ + S. (22) 

jeJVi(t) 


For t > T, rewriting (22) using new variables Xi = Xi — S'dt, we 
have 

Xi=- E sig (Xi-Xj) 71 . (23) 

3<ZNi(t) 

It is proved in 1341 that using (23) , there exists a time T' such that 
Xi = Xj, Vi,j el. As a result we have Xi = Xj, Vi,j e 1, and 
Xi = S, Vt > T + T'. Now, it is easy to see that according to 0 
the optimization goal ( 8 ) is achieved. ■ 

Remark 3.14: Satisfying the conditions mentioned in Assumption 
|3.12| might be restrictive but they hold for an important class of 
cost functions. For example, if the agents’ cost functions are in 
the form of fi(xi,t) = (aiXi + gi(t)) 2 , where the Hessians are 
not equal, the above conditions are equivalent to the conditions 


'As a special case the initial values can be chosen as £j(0) = y: ;! (0) = 

4 >j ( 0 ) = 0 ,Vj e 1. 
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that ||<?i(t)|| 2 i II^(^)II 2 i and II ijiif) II 2 are bounded. This is ap¬ 
plicable to a vast class of time-varying functions, gi(t), such as 
sin(f), e _t cos(f), and tanh(f). 

Remark 3.15: The algorithm introduced in (] 1 6|)-(|20[> just requires 
that Assumptions |3.2| and |3.12| hold. Note that in Assumption |3.2| it 
is not required that each agent’s cost function fi(x,t) has invertible 
Hessian but instead their sum, which is weaker than Assumption 
|3.3| In contrast, for the algorithm 0- not only Assumption |3.3| and 
the conditions mentioned in Remark 13.111 have to be satisfied for 
each individual function it requires the agents’ Hessians 

to be equal. However, in the algorithm the agents just need their 
own positions and the relative positions between themselves and their 
neighbors. In some applications, these pieces of information can be 
obtained by sensing; hence the communication necessity might be 
eliminated. In contrast, in the algorithm - ( 20 ) each agent must 
communicate three variables Wi , q and tpi with its neighbors, which 
necessitates the communication requirement. 


integrating both sides of (26) , we can see that \/fo(x,t) £ £ 2 . 
Now, applying Barbalat’s Lemma [32J, we obtain that V fo(x,t) will 
converge to zero asymptomatically. Then by using Lemma |2.2| and 
under Assumption HD it is easy to see that fg will be minimized, 
where x(t) converges to the optimal trajectory Xg (t). ■ 

The results from Lemma 14.11 can be extended to minimize the 
convex function fi{ x , £)• If Assumption 


3.2 


holds, with 


N N r) rl N rl 

u = - C22 H ^ x ’ ^r 1 ( J2 ot j t v ^ x ’ *) + Y. j t *)) 

2 =1 i= 1 i= 1 

N N 

-(£#<(*, t))(£v/i(s,t)) 

i=1 i= 1 

/ JV N , N 

' 2=1 2=1 ^ 2=1 

N r, N 

2=1 2=1 



IV. Time-Varying Convex Optimization For 
Double-Integrator Dynamics 

In this section, we study the convex optimization problem with 
time-varying cost functions for double-integrator dynamics. In some 
applications, it might be more realistic to model the equations of 
motion of the agents with double-integrator dynamics, i.e., mass- 
force model, to take into account the effect of inertia. Unlike single¬ 
integrator dynamics, in the case of double-integrator dynamics, the 
agents’ positions and velocities at each time must be determined 
properly such that the team cost function is minimized. However, 
there is only direct control on each agent’s acceleration and hence 
there exist new challenges. As a first step, in Subsection |IV-A[ a 
centralized algorithm will be introduced. 


for (24) , the function fi( x ^) * s minimized. Unfortunately, (27) 
is a centralized solution relying on the knowledge of all fi(x,t),i £ 
X. In Subsections |IV-B| and |IV-C| (27) will be exploited to propose 
two algorithms for solving the time-varying convex optimization 
problem for double-integrator dynamics in a distributed manner. 

B. Distributed Time-Varying Convex Optimization Using Neighbors' 
Positions and Velocities 

In what follows, we focus on solving the distributed time-varying 
convex optimization problem 0 for agents with double-integrator 
dynamics 

Xi{t) = Vi(t) Vi(t) = Ui(t), ieT, (28) 


A. Centralized Time -Varying Convex Optimization 

In this subsection, we focus on the time-varying convex optimiza¬ 
tion problem of 0 for double-integrator dynamics 

x(t) = v(t) v(t) = u(t), (24) 


where x,v,u £ R m are the position, velocity, and control input, 
respectively. Our goal is to design the control input u to minimize the 
cost function fo(x, t). In Theorem 4.1[ an algorithm will be proposed 
to solve the problem defined by 3| and (24) . The control input is 
proposed for as 


M = -H 0 1 (x, t)(2L 2^Vfo(x, t)+2^Vfo(x, t)) 
-H 0 (x,t)Vfo(x,t) + ^Hq 1 (x, t)[^Ho(x, t)]Hg 1 (x,t)^ 

(^V/ 0 (tM) + V/o(M))- 


(25) 


Theorem 4.1: Suppose that fa(x,t) satisfies Assumptions 3.1 and 


3.2 Using (25) for (24) , x(t) converges to the optimal trajectory 
Xo(t), the minimizer of ([3}, i.e. lim t _,. 00 [a;(f) — *o(£)] = 0. 

Proof: Define the Lyapunov function candidate W = 
!V/(f(x,f)V/o(iE, t) + i(S'o - v) T {So - v ), where S 0 = 
Ho~ 1 {x,t){-§i ^+ V/o(*,£))■ The derivative of W along 
the system (24) with the control input (25) is obtained as 


W = Vf 0 T (x,t)(Ho( x ,t)v + ^Vf 0 (x,t)) 

+ (So - v) T (S 0 - u) = —V/(£(*, f)V/ 0 (ai, £) 


(26) 


Therefore, W < 0 for Vfo(x,t) 7 ^ 0. Now, having W > 0 
and W < 0, we can conclude that V/o(®, t), So — v £ Coo- By 


where Xi,Vi £ R m are, respectively, the position and velocity, and 
Mi £ R m is the control input of agent i. In this subsection, an 
algorithm with adaptive gains will be proposed, where each agent 
has access to only its own position and the relative positions and 
velocities between itself and its neighbors. The control input is 
proposed for (28) as 

ui = - p(xi — Xj) + a(vi — Vj) 

ieNi 

- PijSgnilixi ~ Xj) + C (vi - Vj)) + <j)i (29) 

jeNi 

Pij = 117(34 - x j) + C (vi - Vj)|li, j € Ni, 

where 

4* = -H-\ Xi ,t){^VU{xi,t) + ±Vfi(xi,t)) 
—Hi(xi,t)Vfi(xi, t) 

+ ^Hr 1 ( Xi ,t)[j t H i (x i ,t)]H- 1 (x i ,t)^ 

fi{xi,t) + v/i(*i,f)), 

where p, a, 7 and ( are positive coefficients, and /3ij is a varying 
gain with /3ij(0) = Pji(0) > 0. Note that 4>i depends on only agent 
i’ s position and velocity. Furthermore all terms in (30) are assumed 
to exist. Define agent i’s position and velocity consensus error as, 
respectively, e Xi = - 4 Y2t=i x i and e U = Vi - ^ Y)f=i v £- 

Let X = [xi ,xj, ...,xj,] and V = [v[, mJ, ..., vJf\ T . Define the 
consensus error vectors for position and velocity as 

ex{t) = (n 0 I m )X, e v (t) = (n 0 I m )V. (31) 
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As discussed in Subsection III-B 


0 ,ey(t) = 0 if and only if Xi = 

<i>^ or.o 2 '\..,oX] r . 


it is easy to see that ex(t) = 
Xj,Vi = Vj, Vi,j £ X. Also let 


Assumption 4.2: With <j>i defined in P0| >, there exists a positive 
constant (f> such that || <j>i — <f>j || 2 < cf>, Vi,j £ X, and Vt. 

In Proposition [2] we will show that the agents reach consensus using 
(29). Then this result will be used in Theorem |4.3| to show that the 
agents minimize the team cost function as t —> oo. 


Proposition 2: Suppose that the graph Q is connected, and As¬ 
sumption | 4.2| and X < A 2 [ L\ hold. The system (28) with the 
algorithm ( 29 ) reaches consensus, i.e, Xi = Xj,Vi = n,, Vi,j £ X, 
as t —> 00 . 


Proof: The closed-loop system (28) with the control input (29) 
can be recast into a compact form as 


(X = V 

<U = ~(L®I m )(pX + aV) + $ (32) 

{ ~{D'®I m )sgn{[D T ®I m ][yX + <:V]), 


where D' is defined in Definition |3.7| and D and L are defined in 
Section H. Now, we can rewrite (32) as 


{ ex = ev 

e v = ~{L ® 7 m )(/ze A - + ae v ) + (n ® 7 m )<f> (33) 

~(D' ® 7 m )sgn([D T ® I m ][yX + CV]). 


Dehne the function 



+ ^EE(fe-« 2 ’ (**) 

2= 1 j£Ni 


where P = 


(cry + p()(L® I m ) -ylmN 
'ylm.N (ImN 

be selected. To prove the positive definiteness of P, we d efine 


and ft > 0 is to 


P = 


{ay + p()(\ 2 [L]I mN ) ylmN 

ylmN (ImN 


By using Lemma |2.l| 


we obtain that P < P . Hence we just need to show P > 0. Now, 


applying Lemma 2.3 P > 0 if ({ay + z[L]In — 7 J /jv > 0, 


which is equivalent to ^ < A 2 [L], 


The time derivative of W along (33) can be obtained as 

W = —yp,e*x(L ® I m )e x + ey{yI mN - a(L ® I m )e v 
- (yex + (e v ) T ([D' ® 7 m ]sgn([73 T ® 7 m ][ ye x + (ev]) 

+ {'yex + (ev) T (U ®I m )$ + -Y / J2 (Pv ~ fife 

i= 1 jeNi 

= -'Ute x (L ® Im)e X + ey{yI mN - a(L ® I m )e v 

-.^EE fe l|7(ejc 4 - e Xj ) + ({e Vi ~ 

2=1 jeNi 
^ N N 

+ nJ 2J2 - e x ) + - ey j )] & 

i= 1 3 = 1 

1 N 

+ 2 EE <Pv - fife 

2=1 jeNi 

< -ype x {L ® Im)e x + e^(7 ImN - a(L ® I m )e v 

1 N 

fe \H e *i - ex i ) + - e x)\\i 

2=1 jeNi 
^ N N 

+ 2jvEEH ex < _ex i) + ^ — e ^)Hi Hi ~ <t>j\\ 

2 = 1 j = 1 

1 N 

+ “ fi l|7(ex, - e Xj ) + C(e Vi - 

*=1 jeNi 


< -ypex{L ® I m )ex + eyipImN - a(L ® 

+ ( e ^-f)i;i;i|7(«x,-«x,.) + C(ev, 


2=1 jeNi 


ev, 


where the l ast i nequality is obtained because Q is connected and 
holds. The term ey{yImN — a(L ® I m )ev < 0 if 


4.2 


Assumption 

ylN — a(L < 0. By applying Lemma 0 we know that ylN — 
a(L < 0 if 7 — a(A 2 [L\ < 0. Select f) such that j3 > tXzPf. 
Using an argument similar to (T3), we have 


W <( (JV 4 ^ h(e Xi - e Xj ) + ((e Vi - 

i=i jeNi 

=( ( iV 2 -j)(ye x +(e v ) T 

^[D 1 ® 7 m ]sgn([79 r ® Im^yex + Cev])^ 

< ( ^ N ~ P)\\(D T ® P^i-yex + (ev)\\ 

2 II Hi 

< ( (jV : 1} ^ - P)VHL] ll(7ex + Cev)|| 2 < 0 , 

(35) 


whe re in the last inequality the fact that L = DD T and Lemma 


2.1 have been used. Therefore, having W > 0 and W < 0, we can 
conclude that e x ,ev £ Coo- By integrating both sides of (35), we 
can see that e x ,ev £ £ 2 - Now, applying Barbalat’s Lemma (32) , 
we obtain that e x and ey will converge to zero asymptotically and 
hence the agents reach consensus as t —3 00. ■ 

Theorem 4.3: Suppose that the graph Q is connected, and Assump¬ 
tions [5T| [13] and [A2] hold. If Hi(xi,t) = Hj(xj,t), Vi,j £ X, and 
-X < \ 2 [L\ hold, by employing the algorithm 1 29 1 for the system 
( |28| >. the optimization goal (8) is achieved. 















Proof: Define the Lyapunov function candidate 

N N 

w = ,(E v /i(*i.*)) T (E 

j=i i=i 

AT AT JV AT 

+ *£>-E^) T (E^-E^ 


(36) 


3 =1 


i=i 


i=i 


j'=i 


where = Hj~ 1 (xj, t) (Jf Vfj(xj, t) + V fj(xj,t)). Note that <5 
is undirected. By summing both sides of the closed-loop system {28) 
with the controller {29), we have Vj = IZ rLi 4*3- The time 

derivative of W along with the system dehned by ( |28| ) and {29) can 
be obtained as 


W =(^2 V fj(xj,t)) T Hj{xj,t)vj + 22 ^V/j(xj,t)) 

j =1 3 = 1 3 = 1 

N N N N 

+ 22 r i - 22 • s i) 7 'E 'j E •‘v) 

i=i j=i j=i j=i 

A/ - AT TV „ 

=(22^ fj(x j ,t)) T + '22^fj{x j , t )) 

3 = 1 3=1 3=1 

N N N N 

••>>,, E s ;) r Vo,. evt 


f=i 


r=i 


r=i 


j=i 


=(22^ fii x jit)) T 222 H i^i^) v j+'22^ t vf j (xj,t)) 


- (22 v * - ^22 s i) T fj{xj,t)). 

3 = 1 3 = 1 3=1 

Now, under the assumption of identical Hessians, we have 

N N Nr, 

W =C22 y fj ( x 3 3)) T {Hj (Xj, t) [^2 + 22 to V fi ’ *)) 


3 = 1 


(37) 


3=1 3=1 

N N N 

- (J2 v j - 22 s ^ T ( H ^ x x t ')i22 vfj( x j,t)]) 

3 = 1 3 = 1 3 = 1 

N N 

= - (E v /f(^- t )) T (E v ^(^’ t ))- 

3 = 1 j = l 

Therefore, W < 0 for JTyLi ^t) ^ 0. Now, having W > 0 
and W < 0, we can conclude that V/, (x 3 ,1), (X^i v 3 ~ 

J2j=\ S:i ) £ £oo- By integrating both sides of {37), we can see that 

we 


YLj—j V f] (x-j . 1) € £ 2 - Now, applying Barbalat’s Lemma 


32 


obtain that XTi=i V fi{x, t) will converge to zero asymptomatically. 
We also have x, = Xj,Vi = Vj. Mi,j G 1 as t —> 00 from 
Proposition [ 2 ] Now, under the assumption that Yl2=i * s 

con vex and using Lemma |2.2| it is easy to see from Assumption 


3.1 


and 11 Vi — Vj 
\\Vfj(xj,t) - Vfi(xi, 


'iVfi(x j ,t)-±Vfi(x i , t)|| a 


and 


for all t. For 

example, for the cost functions fi(xi,t .) = (axt + gi(t)) 2 
introduced in Remark |3.1 1| the boundedness of ||a:; — Xj || 2 
and 11 ttj — Vi 


It) - Vfi(xi, 


'iVf j (x j ,t)-&Vfi( Xi , i)|| a 


\ 9 i(t) - gj(t)\\ 2 are 
\2 


holds for fi(xi, t) = (axi + gi{t)) 

2 and Il5i(i) -flj(t)l| 2 ,Vi and 


hi{t)-gj(t)\\ 2 , \\ 9 i(t)-di(t)\\ 2 and 

bounded. Hence Assumption 4.2 
if \\9i{t) -9j(t)\\ 2 , || 9i(t)-gj 
Mi,j G X are bounded. 

Remark 4.5: The result in Theorem 4.2 can be extended to a 
class of cost functions whose Hessians have the same structure 
rather than being identical under a certain additional assumption. 
Particularly, the assumption that Hi(xi,t) = Hj(xj,t),Mt and 
Mi,j G X, can be replaced with Hi(z,t ) = Hj(z,t), Mz,t and 
Mi, j G X , with an additional assumption that || V/;(xi, t) || 2 and 
||^V/i(xi,f)|| 2 , Vi G X and Vt, are bounded. 

To relax the assumption on Hessians, an estimator-based algorithm 
will be introduced in Subsec tion |IV-C| where the agents can have cost 
functions with nonidentical Hessians. 


C. Estimator-Based Distributed Time-Varying Convex Optimization 
In this subsection, an estimator-based algorithm is designed to 
solve the problem {7) for double-integrator dynamics {28). In this 
algorithm, each agent calculates {27} in a distributed manner. Similar 
to Subsection |III-C| distributed average tracking is used as a tool 
to estimate the unknown variables in {27). Each agent generates an 
estimate of {27}. Then by using the control input Ui(t), each agent 
tracks its estimated signal while reaching consensus. The proposed 
algorithm for the system {28} is given by 


ti= n *22 sgn (wj -Wi), 
jeNiV) 

4>i=P E s S n te~ ?i )> 
ieN+t) 


= & + ipi 


Q — <j>i + 0i 


(38) 


(39) 


Si = ? i i 1 ?i 2 ?ii (wii + wa) - Cii 1 (wi3 + »«) - QiWii (40) 

Mi = - 22 - x j) ai - 22 si s( Wi _ v i) ai + Si > 


jeJVAt) 


leJVi(i) 


where 


and 


0 < a\ < 1, «2 = 


2ai 

ot\ + 1 


(41) 


(42) 


ipi = 


/ X7fi(xi,t ) \ 
§- t Vfi( Xi ,t) 

£ t X7fi(xi,t) 

\wt£vMxi,t)J 


Bi = 


Hi(xi,t) 
f t Hi{xi,t) 


that as t -> 00 , ^2j =1 fi(xj,t) will be minimized, where 
Xi = Xj, Mi,j G X. ■ 

Remark 4.4: In Appendix [A] we show that Assumption |4.2| 
holds if the cost functions with identical Hessians satisfy 
certain conditions such that the boundedness of \\x,. — Xj \\ 2 
for all t guarantees the boundedness of 


and k and p are positive constant coefficients to be selected. Eqs. {38) 
and {39} are distributed average tracking estimators, where the esti¬ 
mated variables Wi and q can be redefined as wf = (w [\,..., wf^) 
and with Wij G K m ,Qfe G R mXm ,Vi G X,j = 

1,..., 4, k = 1, 2. In implementing {40), Q 1 can be projected on the 
space of positive-definite matrices, which ensures that Qi remains 
nonsingular. The initial values of the internal states and <j>i are 
chosen such that the condition 0 

N N 

E^(°) = E^( 0 ) = 0 (43 ) 


3—1 


3=1 


for all t guarantees the boundedness of 


and ||^ 2 -'V/j(asj-,£) - ^2 V/i(a;i,f)|| 2 for all t, if 


hols. 

Assumption 4.6: The coefficients k, and p satisfy the following 
inequalities: k > sup t ||'i/’i|| 00 and p > sup t ||0i|| , Vi G X. 

These assumptions can be satisfied if the graph Q{t) is connected 
for all time, the gradients, the derivatives of the Hessians and 

2 As a special case the initial values can be chosen as £j (0) = <p :i (0) = 
0, Mj G X. 
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gradients, and the partial derivatives of the gradients’ derivatives 
are bounded. Although these assumptions seem restrictive, they can 
be satisfied for many cost functions. For example, for fi(xi,t) = 
( a,iXi +gi(f)) 2 introduced in Remark 3.14 the above conditions are 
equivalent to the conditions that ||g , i(i)|| 2 , ||<?i(t)|| 2 , and ||gi(f)|| 2 
are bounded. 

Theorem 4.7: Suppose that the graph Q(t) is connected for all t 
and Assumptions |3.11 and |3.2| hold. If Assumption |4.6| a nd the initial 
condition © hold, by employing the algorithm (|38|> -© for the 
system {28}, the optimization goal {8} is achieved. 

Proof: Estimator: It follows from Theorem 2 in (33) that 
if k > sup t \\ipi]\oo ,p > sup t 110*11^ , Vi £ I and the graph 
Q(t) is connected for all t, then employing (38) and (39), there 
exists a T > 0 such that for all t > T, ||wi — V’*|| = 

0, |?i - ± 12iLi 0i|| = 0. Note that for t > T, qi is nonsingular 
without projection due to Assumption |3.2| and hence the projection 
operation simply returns qi itself. Now from {40}, for t > T, the 
estimated signal Si satisfies 

N N 

s i = 

i= 1 i=1 

N N f) rl N d 

- ( ]T Hi ( Xi ' 1 ) ) _1 ( at J t V 'fi (- Xi ’ t ) + Yi J t V fi ( X i > t ) ) 

2 = 1 2 = 1 2=1 

/ N N , N \ 

+(E Hi ( Xi ’ *)] _1 E Hi ( Xi ’ f )] E Hi ( Xi > *)i) 

' 2=1 2 = 1 2=1 ' 

N N d 

fi(xi,t )), 

2=1 2=1 

(44) 

which shows that each agent has an estimate of (27), where Si = 
s h Vi,j exyt>T. 

Controller: Note that for t>T, Si = Sj, Vi,j £ X, denoted as 
S. For t > T using {41} for (28) , we have 


v i=- Y1 si %( Xi ~ Xi)" 1 “ - v i) a2 + s - 




ieJVi(t) 


(45) 


For t > T, rewriting © using new variables Xi = Xi — f*. S'dt 
and Vi = Vi — J ^ S'dt, we have 


Vi -~ Y2 s ig (Xi-Xj) 011 - Y2 sig(vi - Vj) a2 . (46) 

ieNi(t) ieJVi(t) 

It is proved in 1351 that using (46), there exists a time T 1 such that 
Xi = Xj, Vi = Vj, \/i,j £ X. As a result we have Xi = Xj, Vi = 
Vj, Vi,j £ X, and Vi = S, V t > T + T'. Now, it is easy to see 
that according to (27) and {44} and Assumption |3.1| the optimization 
goal (8) is achieved. ■ 

Remark 4.8: In the algorithm (38)- ©- it is just required that 
Assumptions |3.2| and |4.6| hold, where Assumption |3.3| does not nec¬ 
essarily hold. However, each agent must communicate two variables 
ipi £ R 4m and 0,: £ R 2mXm with its neighbors, which necessitates 
the communication requirement. On the other hand, for the algorithm 
(29), not only Assumptions [33] and the conditions in Remark |4~4| have 
to be satisfied for each individual function fi(xi,t), it requires the 
agents’ Hessians to be equal. In spite of these restrictive assumptions, 
using {29}. we can eliminate the necessity of communication between 
neighbors when the relative positions and velocities between each 
agent and its neighbors can be obtained by sensing. 

In what follows we will study how to overcome the possible 
chattering effect of implementing the signum function in the algo¬ 
rithm (29). In Subsections |IV-D| and |IV-E[ two continuous control 
algorithms will be proposed to extend ©■ 


D. Distributed Time-Varying Convex Optimization Using Time- 
Varying Approximation of Signum Function 


In this subsection, we focus on distributed time-varying convex 
optimization for double-integrator dynamics (28), where a continuous 
control algorithm based on the boundary layer concept will be 
introduced. Using a continuous approximation of the signum function 
will reduce chattering in real applications and make the controller 
easier to implement. In this algorithm, each agent needs to know 
its own position, velocity and the relative positions and velocities 
between itself and its neighbors. Define the nonlinear function h(-) 
as 


h(z) 



(47) 


where c and e are positive coefficients and z £ R m . The nonlinear 
function h(z) is a continuous approximation, using the boundary 
layer concept [36|, of the discontinuous function sgn(-). The size of 
boundary layer, ee~ ct is time-varying and as t —► oo the continuous 
function h(z) approaches the signum function. The idea of using this 
continuous approximation is borrowed from ©, ©■ 

By replacing the signum function with a continuous approximation 
©. the results presented in Subsection |IV-B| are not valid anymore 
and it is not clear whether the new algorithm works. The reason is 
that the results (and the proofs) in Subsection |IV-B| build upon the 
property of the ideal discontinuous signum function, which switches 
instantaneously at 0, so that it can compensate for the effect of 
the inconsistent internal time-varying optimization signals among the 
agents so that the agents can reach consensus. However, this can no 
longer be achieved by its continuous replacement and further careful 
analysis is needed. The results and the proofs presented in this sub¬ 
section are not just a simple replacement of the signum function with 
its approximation. Here in particular we show that with the signum 
function replaced with the time-varying continuous approximation, 
(47), it is possible to still achieve distributed optimization with zero 
error under certain different assumptions and conditions. The reason 
is that (47) approaches the signum function as t —> oo. 

The continuous control input with adaptive gains is proposed for 



Uj = - p(xi - Xj) + a(vi - Vj) 

jeNi 

- Pd h (T[xi - Xj] + c [Vi - Vj)) + (pi, (48) 

jeNi 

$ij =(7 [xi - Xj] + C[«i - Vj])h{y[xi - Xj] + C[«< - Vj]), 


where p, a, 7 and £ are positive coefficients, /3ij is a varying gain 
with /3ij(0) = (5ji(0) > 0, and (pi is defined as in (29). 

Theorem 4.9: Suppose that the graph Q is connected, and 


j_ tp 

a( a 


< ML] 

(49) 


(50) 


(51) 


hold, where ip > 0 is a parameter to be selected. If Assumptions |3.1[ 


3.3 


and 


4.2 


hold and Hi(xi,t) = Hj(xj,t),Vt and \/i,j £ X, by 

employing the algorithm (48) for the system (28), the optimization 
goal (8} is achieved. 

Proof: Define ex 


and 

T 


ey as m 


(31) and y as y T = 
(yi ,..., yjf) = 7e^ + C e v . with t/i £ R m . Rewriting the closed- 
loop system (28) using (48} in terms of the consensus errors, we 
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have 


ex = ev 

e v = —(L <g> Im){ne x + ae v ) 

( T, jeNl faMvi - Vi) \ 


+ (n ® j m )$. 


\T, jeNN pNjh(y N - yj )J 

Define the function 


W = 


ex 

e v 


ex 

ev 




i=l j£Ni 


where P = 


2.1 


Lemma 


P > 0. Using Lemma 


2.3 


- 2ipyI N + (ay + h()\ 2 [L])I n > 0 

- 2(,ipyI N + C(«7 + AiC)A 2 [L] - ") 2 In > 0 . 


Now, using conditions (|49j and ( |50| l, respectively, we have 
- 2ipylN + (ay + 3 OA 2 [L]I n 

> —2ipyI N + ( Q 7 + MC)7 Jiv = JL (—2ipa( + Qi + ay 2 )I N 

aQ aQ 

> -7(-Cm + Cp + ay)I n = — > 0, 

aC C 

which guarantees that the first inequality in ( |54| > holds. Applying 
similar procedure, we have 

- 2^ipyI N + £(ay + nQX 2 [L] - y 2 I N 

> -2£ipyI N + (ay + fj,()-I N - y 2 In 
a 
C*y 

= -— (— 2 aip + [j,)In > 0 . 
a 

Hence W is positive definite. 

The time derivative of W along 0 can be obtained as 

W = -yyex(L ® I m )e x + ey(yI rnN - a(L <g> I m )e v 
( Sjgjv! Pij h (yi — yj) \ 


- 2tpyexe v - y T 


\E je N N PNjh(yN -Vo)) 

i N 

+ y T (n <g> 7 m )T> + - £ (Pij ~ 0)0ij- 


2=1 jeNi 


We rewrite 0 

yj) + v T ( u 

where W 


as W = W - Ef=iE jeN , 

> + jEillE jeNiiPij - 0)0 

ex \ pf e x 
ev ) \ ev 

—\iy(L ® Im) —ipylmN 

— IpylmN ylmN - a((L <g> I, 


is connected, we have 
1 N 


w = w- ~22 £ P'jfa- 


2 = 1 j£Ni 


(52) 


^ IV JV 1 TV 

+ £ £(^ “ %)& + 2 £ I] (Pij - 0)(yi - yj)KVi - Vi) 

2=1 1 = 1 2 = 1 j(zNi 

N N 

= W + 9iv £ £(«< ~ w)(^i - 


(53) 


<=l 3=1 

0 N 

- 2^ £ (s/» - yj)Hvi - yj) 

2=1 j(zNi 

N N 

~ W+ 2 ^ “ %ll 2 II & - ' 


-2ipyI Nm + (try + 3 <)(X ® Im ) 7 ImN 

ylmN CImN 

and 0 > 0 is to be selected. To prove the 

positive definiteness of P, we also define P = 
-2ipyI mN + (cry + 3iC)(A 2 [L]/ m iv) 7 /miv 
7 ImN C,ImN 

we obtai n tha t P < P. Hence we just need to show 
we know that P > 0 if 


i=i 3=1 


-f££ 

2 = 1 j£Ni 


yi-vj ll 2 


By using 


<W + 


-§££ 

2 = 1 j(zNi 


(N- 1)£ 


£ £ 111/* — %'l 

2=1 j£Ni 

hi - Vj II 2 


2, . . (54) 


where in the last inequality Assumption 4.2 is used. Selecting a 0 


such that 0 > 


(N-l)<t> 


, we obtain 


^<^+f£ £ 

i = l jeNi 


(56) 


If we can show IT’ < —%j>W (or equivalently W + tpW < 0), then 
knowing that e~ ct —> 0 as t —> 00 , Lemma 2.19 in |!39J implies that 
the system (|52l is asymptotically stable. Note that W + ipW = 


(—731 + ayip + (iiip)L - 2ip 2 yI N 

0 


0 

(7 + (iP)I N - a(L 


Applying Lemma [27] we obtain that is negative definite if 

(—731 + ayip + Ciu.ip)L - 2il> 2 yI N < 0 
(7 + CVO-fjv - a(L < 0. 


(57) 


(58) 


To satisfy the first condition in ( |58[ >, we just need to show — 7/1 + 
ayip + < 0. Using conditions ( |50| ) and 0 . we have —731 + 

aytj) + < —731 + ^ + < —731 + ^ ^ < 0. To 

satisfy the second condition in {58), we have (7 + C,i!>)In — aC,L < 
(7 + C^)/tv — a(,\ 2 {L\lN < 0, where Lemma 2.1 and condition 
{49) are employed, respectively. Hence, W < —ipW holds and the 
agents reach consensus as t —> 00 . Now, similar to the proof of 
Theorem |4.3| if Hi(xi,t) = Hj(xj,t),Vt and Vi,j £ X, it can be 


shown that E =i ^fj{ x jit) w iU converge to zero asymptomatically. 


( 55 ) Now, under Assumption 3.1 and the assumption that E,-i fi( x it) 


PijVMVi - 
ij’ 

and P 


. Because the graph Q 


is convex, Lemma 2.2 is employed. Using the fact that Xi(t) 
Xj(t),Vi,j £ X as t —> 00 , it is easy to see that the optimization 
goal is achieved. ■ 

Remark 4.10: It is worth mentioning that if we have ^ < X 2 [L\, 
there always exists a positive coefficient t/> such that conditions {49)- 
{U) hold. However, selecting ip based on conditions {49)-{5l} affects 
the convergence speed, where by having a larger ip the agents reach 
consensus faster. To satisfy conditions {49) and it is sufficient 
to have 2 A 2 M a > ^ > ip (e.g, selecting a large a and choosing 
proper 7 , £ and ip). It can also be seen that selecting a large enough 
37 {50) can be satisfied. 
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Remark 4.11: The results in Appendix can be modified for 
Theorem |4.9[ where we can show that Assumption |4.2| holds under 
the same conditions mentioned in Remark l4~4l 


E. Distributed Time-Varying Convex Optimization Using Time- 
Invariant Approximation of Signum Function 

In this subsection, our focus is on replacing the signum function, 
with a time-invariant approximation 


h(z) = 


-I- e 


(59) 


where e > 0. Here, the boundary layer e is constant. Employing (59}, 
instead of © in the control algorithm ( |48| ) makes the controller 
easier to implement in real applications. The trade-off is that the 
agents will no longer reach consensus, which introduces additional 
complexities in convergence analysis. Establishing analysis on the 
optimization error bound in this case is a nontrivial task, which 
is introduced in this subsection. The reason that the time-invariant 
continuous approximation (U cannot ensure distributed optimization 
with zero error is that the global optimal trajectory is not even an 
equilibrium point of the closed-loop system whenever a time-invariant 
continuous approximation is introduced. It is worthwhile to mention 
that if the signum function were replaced with a different time- 
invariant continuous approximation other than (59}, there would be 
no guarantee that the same conclusion in this subsection would still 
hold and further careful analysis would be needed. 

Theorem 4.12: Suppose that the graph Q is connected, Assump¬ 
tions [TT|[33] and [42] hold and the gradients of the cost functions can 
be written as V fi(xi,t) = axi + gi(t), Vi £ X, where a and gi(t) 
are, respectively, a positive coefficient and a time-varying function. 
If conditions (49}- (5j} hold, using (48} with h(-) given by (59} for 
(28), we have 


lim 


JV N 


l N 


(60) 


where x* and v* are the position and the velocity of the optimal 
trajectory, respectively. In addition, the agents track the optimal 
trajectory with bounded errors such that as t —4 oo 


\\Xi — X U < 


|Vi - V || 2 < 


4>N(N - l) 2 e 

TiMmin [P] 

fN(N - l) 2 e 
4V4min [-P] 


(61) 


Vi ex, 


where P is defined after (53} . 

Proof: The proof will be separated into two parts. In the first 
part, we show that the consensus error will remain bounded. Then 
in the second part, we show that the error between the agents’ states 
and the optimal trajectory will remain bounded. 

Define the Lyapunov function candidate W as in (53} Similar to 
the proof in Theorem |4.9[ with hf) given by (59} instead of (47) , we 
obtain W < W+ f YZi T, j€Ni « < ~i>W+ §N(N-l)t, where 
P is selected such that {3 > Then we have 0 < W(t) < 

(1 — e - ^*) + W(0)e~^ t . Therefore, as t — 4 oo, we have 


ex 

e v 


w,[P]<ir=( “ r 

^ 0N(N-l)e 


ex 

ev 


2 


Now, it can be seen that there exists a bound on the position and 
velocity consensus errors as t —4 oo, that is, 


1 N 

xi - y Xi 

N ^ 3 



j = 1 

2 V 

1 N 

Vi -> Vi 

N ^ 3 

A 

< v/ 

3 = 1 

2 V 


I <j>N(N - l) 2 e 
4'iMmin [P] 

/ 4>N(N — l) 2 e 
4 lMmin[R] ’ 


(62) 


where it is easy to see that by selecting larger tf> satisfying conditions 
(49}-(51}, the error bound will be smaller. 

In what follows, we focus on finding the relation between the 
opti mal trajectory and the agents’ states. According to Assumption 


3.1 


and using Lemma 


2.2 


we know JV =1 V fj(x* ,t) = 0. Hence, 
under the assumption of V/; (xi, t ) = axi + gi(t), the optimal 
trajectory is 




L 9j 


N ■ 

— L,=i 9j 


(63) 


No No 

Similar to the proof of Theorem |4.3| we can show that, regard¬ 
less of whether consensus is reached or not, it is guaranteed that 
E| V fj{xj,t) will converge to zero asymptomatically. As a result, 


9JV 

J i =i 

we can 


• >;, .«./ 


Xi —4 -and ^2 j=1 Vi 

conclude (60). According to 




By 

it follows that 


we have ^ 
using 

(61} holds. 

Remark 4.13: Using the invariant approximation of the signum 
function (59}. instead of the time-varying one (47} , makes the 
implementation easier in real applications. However, the results show 
that the team cost function is not exactly minimized and the agents 
track the optimal trajectory with a bounded error. It also restricts 
the acceptable cost functions to a class that takes in the form 
Vfi(xi,t) = oxi 

Remark 4.14: The results in Appendix can be modified for 
Theorem |4.9| where under the assumption of V fi(xi,t) = 
oxi + it is easy to show that Assumption 

II 9i(t) ~ 9j(t)\\ 2 , II 9 i(t) - gj(t)\\ 2 and ||&(t) - & 

Vi,j £ X, are bounded. 

Remark 4.15: The algorithms introduced Subsections |III-B| |III-C| 
[Tv^b|[T\^di and |IV-E| are still valid in the case of strongly connected 
weight balanced directed graph Q. In our proofs, L can be replaced 
with symmetric matrix L+L T as x T Lx = \x T (L+L T )x. Since Q is 
strongly connected weight balanced, L + Lr is positive semidefinite 
with a simple zero eigenvalue. Note that applying the introduced 
algorithms for directed graphs, we need to redefine A 2 as the smallest 
nonzero eigenvalue of L + L T . 


4.2 


holds, if 
,Vf and 


V. Distributed Time-Varying Convex Optimization with 
Swarm tracking behavior 

In this section, we introduce two distributed optimization algo¬ 
rithms with swarm tracking behavior, where the center of the agents 
tracks the optimal trajectory defined by (7} for single-integrator 
and double-integrator dynamics while the agents avoid inter-agent 
collisions and maintain connectivity. 


A. Distributed Convex Optimization with Swarm Tracking behavior 
for Single-Integrator Dynamics 

In this subsection, we focus on the distributed optimization prob¬ 
lem with swarm tracking behavior for single-integrator dynamics g. 
To solve this problem, we propose the algorithm 


Ui(t) 


fi s gn( 

jeNilt) 


dVjj 

dxi 


) + 4>i : 


(64) 
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where Vij is a potential function between agents i and j to be 
designed, /3 is positive, and <f>i is defined in & In {64}, each agent 
uses its own position and the relative positions between itself and 
its neighbors. It is worth mentioning that in this subsection, we 
assume each agent has a communication/sensing radius R, where 
if \\xi — Xj\\ 2 < R agent i and j become neighbors. Our proposed 
algorithm guarantees connectivity maintenance in the sense that if the 
graph 0(0) is connected, then for all i, 0(f) will remain connected. 
Before our main result in this subsection, we need to define the 
potential function Vij. 

Definition 5.1: © The potential function Vij is a differentiable 
nonnegative function of \\xi — Xj\\ 2 , which satisfies the following 
conditions 

1) Vij = Vji has a unique minimum in \\xi — Xj || 2 = dij , where 
dij is the desired distance between agents i and j and R > 
maxij dij. 

2) Vij —> oo if \\xi — Xj || 2 —> 0. 

3) Va = c, where c is a constant. 

= 0 ||a:;(0) — Xj (0)|| 2 > R, \\xi — Xj || 2 > R, 
—3 oo ||xi(0) — aij(0)|| 2 < J?, laii — Xj\\„ —¥ R. 

The motivation of the last condition in Definition 5.1 is to maintain 
the initially existing connectivity patterns. It guarantees that two 
agents which are neighbors at t = 0 remain neighbors. However, 
if two agents are not neighbors at t = 0, they do not need to be 
neighbors at t > 0 (see [20}). 

Theorem 5.2: Suppose that graph 0(0) is connected, Assumptions 
1 3.1 1 and [33] hold and the gradient of the cost functions can be written 
as V fi{xi,t) = <JXi + gi(t), Mi £ T , where a and gfit) are, 
respectively, a positive coefficient and a time-varying function. If 
(3 > 110*11 i i Vi £ I, for the system 0 with the algorithm {64}, the 
center of the agents tracks the optimal trajectory while the agents 
maintain connectivity and avoid inter-agent collisions. 

Proof: Define the positive semidefinite Lyapunov function can¬ 
didate 

N N 

i= 1 3=1 


4) 


d(\\xi~x 

dVii 


9(||cci— X 


obtained 

■yN 


y ~yl\ 
i=1 = 


as 

dVi. 


w 

Xi, where 


The time derivative of W 

( dV ij j. I 9V ij j. .) 

2 Z^i = l l^j = l t dxi x- g x . Xj) 

in the second equality, Lemma 3.1 in 
rewriting W along the closed-loop system {64} and {!}, we have 


3=1 dx 

201 has been used. Now, 


NN N 


i=l j=l 


3=1 


dxi 


N N ~ N 

+£XS^X 


i=l j=l 


Y 9Vij 


3=1 


dx-i 



It is easy to see that if (3 > || II i , Mi £ X, then W is negative 
semidefinite. Therefore, having W > 0 and W < 0, we can conclude 
that I'jj G oo■ Since \j is bounded, based on Definition ED 
it is guaranteed that there will be no inter-agent collision and the 
connectivity is maintained. 

In what follows, we focus on finding the relation between the 
optimal trajectory and the agents’ positions. Based on Definition |5.1| 
we can obtain that 

dVij dVji dVij 

- 5 — = — = —• * 66 ) 

OXi C/Xj, C/Xj 

Now, by summing both sides of the closed-loop system {7} with the 
control algorithm {64}, for i£lwe have XE = i *3 = X^j = i ' v * ve 
also know that the agents have identical Hessians since it is assumed 


that V f t ( Xi , t ) = crxi + Qi ( t ). Now, similar to the proof of Theorem 
|4.3[ regardless of whether consensus is reached or not, we can show 
that ^fji x j j t) w iH converge to zero asymptomatically. Hence 


we have J2f = i x i 


2.2 


3 = 1 

and under Assumption 


3.1 


L . On the other hand, using Lemma 
know = 0. 


Hence, the optimal trajectory is x* = 


~ £j = l 9j 


NcT 


. This implies that 


jt 77, Xi -> i‘, where we have shown that the center of the agents 
will track the team cost function minimizer. ■ 

Remark 5.3: In Appendix |B] it is shown that a constant (3 can 
be selected such that /? > ||0»||j ,Mt and Mi £ X, if ||3i(f)|| 2 an d 
||fli(t)|| 2 ,Vi and Mi £ X, are bounded. In particular, it is shown 
that such a constant f3 can be determined at time t = 0 by using 
the agents’ initial states and the upper bounds on ||<7i(f)||., and 
\\gfit)\\ 2 ,Mt and Mi £ X. 


B. Distributed Convex Optimization with Swarm Tracking Behavior 
for Double-Integrator Dynamics 

In this subsection, we focus on distributed time-varying optimiza¬ 
tion with swarm tracking behavior for double-integrator dynamics 
{28}. We will propose an algorithm, where each agent has access to 
only its own position and the relative positions and velocities between 
itself and its neighbors. We propose the algorithm 

Ui{t)=- sgn (Vi - Vj) + <f>i, (67) 

3etv ; (t) 1 jeNiit) 


where V tJ is defined in Definition |5.1[ (3 is a positive coefficient, and 
fii is defined in {29}. 

Theorem 5.4: Suppose that the graph 0(0) is connected, Assump¬ 
tions ED and |3.3| hold and the gradient of the cost functions can be 

written as Vfi(xi,t) = axi + gi{t), Mi £ X. If j3 > 11 ,nbJ ’ rl l* 1 * !!. 

_ _ 

holds, for the system {28} with the algorithm {67}, the center of 
the agents tracks the optimal trajectory, the agents’ velocities track 
the optimal velocity, and the agents maintain connectivity and avoid 
inter-agent collisions. 

Proof: Writing the closed-loop system {28} with the control 
algorithm {67} based on the consensus errors ex and ey defined 
in {3T}, we have 


ex = 
ey = 


ey 


—a(L(t) 0 Im)e v - f3{D{f) i 

/ v dVlj \ 

Z—/j£Ni dex. 


i 7 m )sgn([D T (f) 0 Im\ev) 


& V Nj 

\2-^jeN N dex N J 


+ (n 0 / m )$. 


( 68 ) 

Define the positive semidefinite Lyapunov function candidate W = 
J2iLi ^*3 + |eyey. The time derivative of W along {68} 

can be obtained as W = \ Ii Ef=i (§^~ ev i + ^~ e v) + 
eyey. Using Lemma 3.1 in |20) , W can be rewritten as 

W = - aey(L{t) 0 I m )e v - /3ey(D(t) 0 7 m )sgn([D T (t) 0 I m ]e v ) 

4- ey(Tl 0 

Using a similar argument to that in l |13[ , we obtain that if 
/3-y/A 2 [X,(t)] > || (n 0 J m )$|| 2 , then W is negative semidefinite. 
Therefore, having W > 0 and W < 0, we can conclude that 
Vj, e v £ Coo. By integrating both sides of W < —aey(L(t) 0 
X 7 ,)ey, we can see that e„ £ £ 2 - Now, applying Barbalat’s Lemma 
|32| , we obtain that ey converges to zero asymptotically, which 
means that the agents’ velocities reach consensus as t —> 00 . Since 
Vij is bounded, it is guaranteed that there will be no inter-agent 
collision and the connectivity is maintained. 
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In the next step, using ( |66| ), by summing both sides of the closed- 
loop system ( |28|) with the control algorithm ( |67| ) for i £ X, we have 

E 


= Ej=i Now, if the team cost function Ei=i 
is convex and V ,fi(xi,t) = ax j + qj(t), applying a procedure similar 


3=1 


to the proof of Theorem 4.12 we can show that E =i V/, (xj , i) 


will converge to zero asymptomatically and ( |60| ) holds. Particularly, 
we have shown that the average of agents’ states tracks the optimal 
trajectory. Because the agents’ velocities reach consensus as t —> oo, 
we have that Vi approaches v* as t —> oo. ■ 

Remark 5.5: The assumption B > can be interpreted 

F \ F 
as a bound on the difference between the agents’ internal signals. 

Using the fact that \ 2 [L(t)] is lower bounded above 0. t here always 


exists a /? satisfying B > ^ ^ Assumption 


to satisfy Assumption 4.2 with Vfi(xi,t) = ax 


4.2 holds. Here, 

_ Vi £ 

X, the boundlessness of \\gi{t) — gj{t)\\ 2 ,\\gi{t) — gj(t)\\ 2 , an d 
|| gi{t) — gj(t)\\ 2 ,Mt and Mi, j £ X, is sufficient. 

Remark 5.6: The algorithms proposed in Subsections |III-C| and 
|IV-C| and Section [V] are provided for time-varying graphs. For 
algorithms introduced in Subsections |III-B| |IV-B[ |IV-D| and |IV-E[ 
the current results are demonstrated for static graphs. However, the 
results are valid for time-varying graphs if the graph Q(t) is connected 
for all t and a sufficiently large constant gain is used, instead of the 
time-varying adaptive gains. In particular, the constant gain /? should 

satisfy B > ! To select such a B, we need to know <j>, 

1 V a 2[£W1_ r 

defined in Assumption |3.8| (or Assumption |4.2| in the case of double¬ 
integrator dynamics), and Bx (and Bv in case of double-integrator 
dynamics), defined in Appendix [A] 

Remark 5.7: All the proposed algorithms are also applicable to 
non-convex functions. However, in this case it is just guaranteed that 
the agents converge to a local optimal trajectory of the team cost 
function. 


VI. Simulation and Discussion 

In this section, we present various simulations to illustrate the 
theoretical results in previous sections. Consider a team of six agents. 
The interaction among the agents is described by an undirected 
ring graph. The agents’ goal is to minimize the team cost function 
Ei=i /*(*<>*)• where Xi = ( r Xi ,r yi ) T is the coordinate of agent i 
in 2D plane, subject to Xi = Xj. 

In our first example, we apply the algorithm for single¬ 
integrator dynamics Q- The local cost function for agent i is chosen 
as 

fi(xi,t) = (r Xi - isin(f)) 2 + ( r Vi - icos(f)) 2 , (69) 


where it is easy to see that the optimal point of the team cost function 
creates a trajectory of a circle whos e cen ter is at the origin and radius 
is equal to ^ • For {69}, Assumption 3.3 and the conditions for agents’ 


cost functions in Remark [3.1 l| hold. In addition they have identical 
Hessians and the team cost function is convex. Bij( 0) = Bji{ 0) are 
chosen randomly within (0.1,2) in the algorithm The trajectory 
of the agents and the optimal trajectory is shown in Fig. [T] It can be 
seen that the agents reach consensus and track the optimal trajectory 
which minimizes the team cost function. 

In the case of double-integrator dynamics, we first give an example 
to illustrate the algorithm {29} for {28} with the local cost functions 
defined by {69}. Choosing the coefficients in {29} as g = 5, a = 
12, 7 = 5,C = 12, and /3ij (0) = Bji{ 0) randomly within (0.1,2), the 
agents reach consensus and the team cost function is minimized as 
shown in Fig. [2] 

In our next example, we illustrate the results obtained in Subsection 
|IV-C| where it has been clarified that the algorithm {38} -{41} can be 



Fig. 1. Trajectories of all agents along with the optimal trajectory using the 
algorithm {9} for the local cost functions {69} 



Fig. 2. Trajectories of all agents along with the optimal trajectory using the 
algorithm {29} for the local cost functions {69} 


used for local cost functions with nonidentical Hessians. Here, the 
local cost functions 

fi(xi,t) = (^4- - sin(t )) 2 + (— - cos(t)) 2 , (70) 

l l 

will be used, where Hi(xi,t) = -§^ 2 , Vi £ X. It can be obtained 
that the team cost function’s optimal trajectory creates a circle whose 
center is at the origin and radius is equal to 1.64. The algorithm {38} 
-{41} with k = 12 ,p = 2, a\ = 0.1, and «2 = 77 is used for the 
system {28}. Fig. [3] shows that the team cost function is minimized. 

In our next example, the results in Subsection |IV-E| is illustrated, 
where the invariant approximation of the signum function is em¬ 
ployed. Here, the algorithm {48} with h(-) given by {59} is used 
to minimize the agents’ team cost function for local cost functions 
defined as {69}. The coefficients are chosen as g = 5, a = 10,7 = 
5, C = 5, e = 2 and Bij{ 0) = Bji{ 0) randomly within (0.1,2). Fig. [ 4 ] 
shows the agents’ trajectories along with the optimal one. It is shown 
that the agents track the optimal trajectory with a bounded error. 

In our last illustration, the swarm tracking control algorithm {67} 
is employed, where the local cost functions are defined as 

= (r Xi + 2i S1 °^'^ ) 2 + {r Vi + isin(O.lt)) 2 . (71) 

In this case, we let R = 5. The parameter of {67} is chosen as B — 20. 
To guarantee the collision avoidance and connectivity maintenance, 
the potential function partial derivative is chosen as Eqs. (36) and 
(37) in (20| , where dij = 0.5, Mi,j. Fig. 0 shows that the center of 
the agents’ positions tracks the optimal trajectory while the agents 
remain connected and avoid collisions. 

VII. Conclusions 

In this paper, a time-varying distributed convex optimization prob¬ 
lem was studied for continuous-time multi-agent systems, where the 
objective was to minimize the sum of the local time-varying cost 
functions. Each local cost function was only known to an individual 
agent. Control algorithms have been designed for the cases of single¬ 
integrator and double-integrator dynamics. In both cases, as a first 














































14 



Fig. 3. Trajectories of all agents along with the optimal trajectory using the 
algorithm < |38) -ED for the local cost functions ED 



Fig. 4. Trajectories of all agents using the algorithm {48} and ED for the 
local cost functions |69) 


step, a centralized approach has been introduced to solve the opti¬ 
mization problem for convex time-varying cost functions. Then this 
problem has been solved in a distributed manner and a discontinuous 
algorithm with adaption gains has been proposed, where it was pos¬ 
sible to rely on only local sensing. To relax the restricted assumption 
imposed on the feasible cost functions, an estimator based algorithm 
has been proposed, where the agents used dynamic average tracking 
as a tool to estimate the centralized control input. However, the 
necessity of communication between neighbors was the drawback of 
the estimator based algorithm. Then in the case of double-integrator 
dynamics, we have focused on extending our proposed algorithms to 
improve them for real applications. Two continuous algorithms have 
been proposed which employed continuous approximations of the 
signum function. The first continuous algorithm used a time-varying 
approximation of the signum function, where we have shown that the 
team cost function was minimized and the agents reached consensus. 
In the second continuous algorithm, a time-invariant approximation of 
the signum function has been used which was easier to implement. 
The trade-off was that there existed a bounded error between the 
agents and the optimal trajectory. To add the inter-agent collision 
avoidance capability into our algorithms, two distributed convex 
optimization algorithms with swarm tracking behavior have been 
proposed for single-integrator and double-integrator dynamics. It has 
been shown that for both cases, the connectivity of the agents was 
maintained while the agents avoided inter-agent collisions and the 
center of the agents tracked the optimal trajectory. 
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Appendix A 

In this appendix an explanation is given on how Assumptions 


and 


3.8 and |4.2| can be, respectively, satisfied in Theore m |3.9| and 
4.3| We focus on the more involved case in Theorem |4.3| while 
a similar argument holds in the case in Theorem |3.9| Here we 
show that Assumption |4.2| holds if the cost functions with iden¬ 
tical Hessians satisfy certain conditions, referred as Condition (*) 
for convenience, such that the boundedness of \\xi(t) — Xj{t)\\ 2 


and || Vi(t) — Vj(t )\\ 2 for all t guarantees the boundedness of 


d z 

dt 2 


tV fj{xj,t) - fi(xi,t) || 2 , Vf and Vi, j € X. 

This is ensured by Condition (★). Denote the upper bounds 
on || Xi(t) — Xj(t) || 2 and \\vi(t) — Vj(t)\\ 2 ,Vf and Mi,j £ X, 
as, respectively, and fi v . It is easy to see that if there exist 
constant /3 X and /3 V , there exists a constant <j>, which in turn 
guarantees the existence of bounded /3, where /3 > . 

2) Our proof will be completed if we can show that there exist 
constant /3 X and /3 V . In the remaining, we will show that not 
only there exist constant f5 x and f3 v , but also it is sufficient to 
determine these constants using the agents’ initial states. Two 
conservative /3 X and /3 V are selected using the initial states as 


I3 X =2 


m\„ 

NX, 


x[P] 

4 PI 


3 = 1 


£>i(0) -Vj( o)) 
i =i 


Pv =2 


mA m ax[D] 


, , (ma 
N\ min [P] y i 


£Oi(0) - Xj(0)) 
3=1 


(72) 


+ max 


£(«i(0)-«i(0)) 

3=1 


-7, 


where 7 is a positive constant, Amm[-P] > 0, and A m ax[-P] > 
0 with P defined in d34l. Now. we will show that 


\\xi(t) - Xj(t)\\ 2 < fi x , \\vi(t) - Vj{t)\\ 2 < /3„, \/i,j £ 1 

and Vf. 

3) We know that for W defined in (|34j and for ft selected based 
on the introduced constants ft x and /3„, we have Wi t= o < 0. 
We will use a contradiction approach to show that such /3 
ensures W < 0,Vf. Assume that there exists a time f = f e at 
which W becomes positive, i.e, W\t=t e > 0. By recalling the 
selected conservative j3, this is only possible if one or more 
of the constants /3 X , and fi v do not exist. This means that 
there exist two agents k and l such that ||a:fc(t e ) — xi(tZ)\\ 2 > 
1 3 X or \\vk(t e ) — vi(t e )\\ 2 > fi v . Let us first suppose that 
\\xk(t e ) — Xi(fe )|| 2 > /3x- Note that that W(t) < 0, Vf £ 
[0,f £ ), which means that W(t) < VU(0), Vf £ [0,f c ). 


Using a 

4 


/ Xmax [ l 1 ] 

A mi „[P] 


it is easy to see that Vf £ 


ex( 0 ) 

ev(0) 


> 


ex (f) 
ev{t) 


[ 0 ,t e ) we have 
. Now, using the 


graph connectivity and the properties of the norms, it is easy 
to show that 


a 0 / mA max [P] , 

p x — 7 = 2 < /- max 

^ ^ N\ min [P] y i 


£040) - Xj(0)) 

3=1 


\\Vfj(xj,t) - V/i(xi,f)|| 2 , \\ Tt \7fj{xj,t) - MVfi{xi,t)\ 2 and 
j | V fj ( Xj , f) — V fi ( Xi , f) 11 for all f. As a result, if Condition 

+ max 
i 

£M0) - Vj( 0)) 


(*j is satisfied, then Assumption 4.2 holds. 


3 = 1 

00 


I Nm\ max [P] 
Anin [-F*] 


(l|ex(0)|| c 


In particular, we will show that under Condition (*), there always 
exists a finite <j> which can be determined at time f = 0. With (j> 
determined, using the algorithm {29), ||(£) — Xj(t)\\ and ||t7(f) — 
Vj(t)\\,\/t and Vi, j £ X , will remain bounded for all f > 0, which 
implies that Assumption |4.2| holds. We show the argument in four 
steps. 


■IM0)IU>2 


> 2 


Amax f f’l 


NrnA m ax[P] 

A1,, j n [P| 


ex(0) 

e v (0) 


1) With identical Hessians. 

11 vfj : (xj,t)-vM Xi ,t)\\ 2 . 


Assumption |4.2| holds if 


^mm[P] 

> 2 || ejvr (i) || 2 > 2 max 


ex(0) 

e v (0) 


> 2 


e x(f) 

e v (f) 


||V/ J -fe,f)-^V/ i (x i ,f)|| 


(*)- 


3=1 


and 


, Vf and Vi, j £ X , are 


i fpVfj{xj,t) — | ^ 

bounded. Assume that the boundedness of \\xi(t) — Xj(t)\\ 2 
and \\vi(t) — Vj(t)\\ 2 , Vf guarantees the boundedness of 
\\Vfj(xj,t) — V/j(a:i,f)|| 2 , WiVfjixjX)- Hx„t)\\ 2 


> ||acfc(t) — a;((f)|| , Vfc, Z £ X, and V t £ [0, t e ). 

Now, under the assumption of \\xk(t e ) — xi(t e )\\ 2 > /3 X 
and using the selected /3 X in {72), we have 
\\x k {t e ) - xi(t e )\\ 2 - lim t ^ t - ||atfe(f) — a;i(f )|| 2 > 7 . 
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However, lim^- \\xk(t) - xi(t)\\ 2 ^ \\x k (te) - xi(t e )\\ 2 , 
contradicts with the continuity of the agents’ positions 
as mentioned in Remark |3.6| Therefore, we have 
|| £Ci (i) — Xj(t) || 2 < p x , Vi,j £ X and Vf. The same argument 
can be made for showing ||i>i(f) — Vj(t) || 2 < /?„, Vi,j £ X 
and Vf, which is omitted here. We thus conclude that there 
exists no time t = f e , where > 0 . 


For example, to satisfy Assumption |4.2| fo r the local cost function 
= (axi+ gi(t)) 2 defined in Remark [3.11| it is only required 
to satisfy Condition (*). It is easy to see that Condition (*) boils 
down to the boundedness of \\gi(t) — gj(t)\\ 2 , \\gi(t) — gj(t)\\ 2 , and 
\\gi(t) — ijj(t)\\ 2 ,\/t and Vi, jr £ X. Hence the boundedness of 
|| 3 ,:(f)- 3 J (f)|| 2 ,|| 5 i(f)-Si(f)|| 2 , and ||<?i(t) - gj(t)\\ 2 , Vf and 


Vi, j £ X, is sufficient to ensure that Assumption |4.2| holds. 

A similar argument can be done for satisfying Assumption |3.8| 
in Theorem |3.9[ Here for cost functions with identical Hessians, 
Condition (*) is such that the boundedness of \\xi(t) — Sj(f )|| 2 for 
all f guarantees the boundedness of ||V/j(xj,t) — Vfi(xi,t)\\ 2 and 
|| V/j( x,,f) — ^V/i(xi,f )|| 2 ,Vi,j £ X, for all f. As mentioned 

inRemark3.11 for the local cost function .fi(xi, f) = (axi+gi(t )) 2 , 
Condition (★) boils down to the boundedness of |g,(f) — g : j{t)\\ 2 
and ||<jj(f) — c/j (f )|| 2 , Vf and Vi,j £ X. Hence the boundedness 
of ||< 7 i(f) — gj(t)\\ 2 and ||<?i(t) — gj{t) ||., is sufficient to ensure that 
Assumption |3.8| holds. 


Appendix B 

In this appendix, we clarify how the boundedness of (pi,Vi £ X 
and Vf, in Theorem |5.2| can be satisfied. In particular, we show that 
for bounded ||gi(f )|| 2 and ||<ji(f )|| 2 , Vf and Vi £ X, there exists a 
constant ft such that p > 11 Hi, Vi £ X and Vf. The constant /3 
can be determined at time f = 0 using the agents’ initial states and 
the upper bounds on ||<?i(f)||,, and ||Pi(f )|| 0 , Vf and Vi £ X. 

Denote the upper bounds on ||ati(f )|| 2 , ||<ji(t )|| 2 and ||<?i(f )|| 2 ,Vt 
and Vi £ X, as, respectively, P x ,g and g. It is easy to see that if 
there exist constant P x ,g and g , there exists a constant p, where 
P > 1111 x , Vi £ X and Vf. We show the argument in three steps. 

1) It is assumed that ||<?i(f )|| 2 and ||(?i(f )|| 2 , Vf and Vi £ X , are 
bounded. Hence, our proof will be completed if we can show 
that there exists a constant p x . In the remaining, we will show 
that not only there exists a constant p x , but also it is sufficient 
to determine this constant using the agents’ initial states. Define 
Px as 

+ —g + (N — 1)R + 7, (73) 

(7 

2 

where 7 is a positive constant and R is defined in Definition |5.1| 
We know that for W defined in |65j and for p selected based on 
the introduced constants P x ,g and g, we have W\ t =o < 0- We 
will use a contradiction approach to show that such P ensures 
W < 0,Vf. Assume that there exists a time f = f e at which W 
becomes positive, i.e, W\ t=te > 0 . By recalling the selected 
conservative P, this is only possible if there exists an agent 
i such that ||a;i(f e )|| 2 > P x . Note that that W(t) <0, Vf £ 
[0, f £ ), which means that Vij (£) is bounded, Vf £ [0, f £ ), which 
in turn implies that the agents remain connected. Hence, it is 
easy to see that Vf £ [0, f £ ) we have 

\\xpt) - 27 (f) || 2 < (N - 1 )R. (74) 

2) Using W defined in 03 and similar to Theorem |3.9| we have 

W(t) < 0 (no matter consensus is reached or not), which in 
turn implies that V/,(a;,',f) is decreasing. Now, for 

3 — 1 9 




X^(°) 


V fi(xi,t) = axi + <?;(f), defined in Theorem 5.2 and using 
the properties of the norms, it is easy to show that 




i =1 
N 






y>/i(*i(o),o) 


< a 




!>;( 0) 


Using the upper bound g, we have Vf 




< 


^ 2 ^( 0 ) 


2N 

+- 9- 

<J 


(75) 


3) Now, using 03 and 03> it is eas y to see that 


IMf )|| 2 < ^ 


^ 27 ( 0 ) 


H —g + (N — 1 )R, (76) 

a 


Vi G X and Vt G [0,t € ). Now, under the assumption of 
||®*(^e)|| 2 > Px and using the selected /3 X in j73| >, we have 
\\xi(t e )\\ 2 - lim t _^ t - ||xi(f)|| 2 > 7 . However, ||xi(f e )|| 2 ^ 
lim t t - ||xi(t )|| 2 contradicts with the continuity of the agents’ 
positions as mentioned in Remark |3.6| Therefore, we have 
||xi(t )|| 2 < Px, Vi G X and Vf. We thus conclude that there 
exists no time t = t e , where W\ t =t e > 0 . 
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